Image processing apparatus that generates stereoscopic print data, method of controlling the image processing apparatus, and storage medium

ABSTRACT

An image processing apparatus includes a distance information acquisition unit that acquires distance information, associated with an object image included in each of a first image and a second image, and corresponding to a distance in a depth direction of the object, a calculation unit that calculates a movement vector of the object image, at least in a horizontal direction and a vertical direction, based on position information of the object image, and a generation unit that generates stereoscopic print data of the acquired second image, based on the second image, the distance information, and the movement vector, by increasing a depth component of the position information of the object image included in the second image based on at least part of the movement vector. In addition, a storing unit stores the generated stereoscopic print data in one of a second memory and a recording medium.

This application claims the benefit of Japanese Patent Application No.2014-262420 filed Dec. 25, 2014, which is hereby incorporated byreference herein in its entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image processing apparatus, a methodof controlling the image processing apparatus, and a storage medium.More particularly, the present invention relates to an image processingapparatus that generates stereoscopic print data for use in stereoscopicprint processing, a method of controlling the image processingapparatus, and a storage medium.

Description of the Related Art

An image processing apparatus, such as a digital camera, generates printdata for printing a photographed image, while a printing apparatus, suchas a printer, performs print processing based on the generated printdata. Further, when the image processing apparatus performs stereoscopicprint processing for a photographed image, the image processingapparatus generates print data for performing the stereoscopic printprocessing (hereinafter referred to as the “stereoscopic print data”).As a technique for generating stereoscopic print data, for example,there has been known a technique for generating stereoscopic print databased on a photographed image and ranging information of an object inthe photographed image (see Japanese Patent Laid-Open Publication No.2008-244831). A printing apparatus dedicated to stereoscopic printprocessing performs stereoscopic print processing based on the generatedstereoscopic print data, and outputs a three-dimensional printing resultbased on the stereoscopic print data.

In generating the stereoscopic print data, however, informationindicating protrusions and indentations of an object can be easilyacquired based on the ranging information of the object, but it isdifficult to directly acquire information indicating the track ofmovement of an object that is moving (hereinafter referred to as the“moving body”) from the ranging information. This brings about a problemthat conventional image processing apparatuses cannot generatestereoscopic print data that gives a printing result expressing afeeling of lively movement of a moving body.

SUMMARY OF THE INVENTION

The invention provides an image processing apparatus that is capable ofgenerating stereoscopic print data that gives a printing resultexpressing a feeling of lively movement of a moving body, a method ofcontrolling the image processing apparatus, and a storage medium.

In a first aspect of the present invention, an image processingapparatus acquires a first image and a second image, the first image andthe second image being photographed at different times, and eachincluding an object image of an object, and that generates stereoscopicprint data for use in stereoscopic print processing of the acquiredsecond image. The image processing apparatus includes a distanceinformation acquisition unit configured to acquire distance informationassociated with the object image included in each of the first image andthe second image, a calculation unit configured to calculate a movementvector of the object image included in the second image based on thefirst image and the second image, and a generation unit configured togenerate the stereoscopic print data, based on the second image, thedistance information associated with the object image included in thesecond image, and the movement vector.

In a second aspect, the present invention provides a method ofcontrolling an image processing apparatus that acquires a first imageand a second image, the first image and the second image beingphotographed at different times, and each including an object image ofan object, and that generates stereoscopic print data for use instereoscopic print processing of the acquired second image. The methodincludes the steps of acquiring distance information associated with theobject image included in each of the first image and the second image,calculating a movement vector of the object image included in the secondimage based on the first image and the second image, and generating thestereoscopic print data, based on the second image, the distanceinformation associated with the object image included in the secondimage, and the movement vector.

In a third aspect of the present invention, a non-transitorycomputer-readable storage medium stores a computer-executable programfor executing a method of controlling an image processing apparatus thatacquires a first image and a second image, the first image and thesecond image being photographed at different times, and each includingan object image of an object that generates stereoscopic print data foruse in stereoscopic print processing of the acquired second image. Themethod includes the steps of acquiring distance information associatedwith the object image included in each of the first image and the secondimage, calculating a movement vector of the object image included in thesecond image based on the first image and the second image, andgenerating the stereoscopic print data, based on the second image, thedistance information associated with the object image included in thesecond image, and the movement vector.

According to the inventions, it is possible to generate stereoscopicprint data that gives a printing result expressing a feeling of livelymovement of an object.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of a digital camera as an imageprocessing apparatus according to an embodiment of the invention.

FIG. 2 is a diagram useful in explaining movement information calculatedby a movement information-calculating section appearing in FIG. 1.

FIG. 3 is a flowchart of a movement information calculation processperformed by the digital camera shown in FIG. 1.

FIGS. 4A to 4G are diagrams useful in explaining the movementinformation used in the movement information calculation process in FIG.3. More specifically, FIG. 4A shows image information of an n-th image,FIG. 4B shows the image information that is divided into a plurality ofblocks, FIG. 4C shows a distance from a photographing section to anobject, FIG. 4D shows an object area, FIG. 4E shows gravity centerposition information of image information of an n−1-th image, FIG. 4Fshows gravity center position information of the image information ofthe n-th image, and FIG. 4G shows the movement information of theobject.

FIG. 5 is a flowchart of a stereoscopic information calculation processperformed by the digital camera shown in FIG. 1.

FIG. 6 is a diagram useful in explaining stereoscopic information usedin the stereoscopic information calculation process in FIG. 5.

FIG. 7 is a schematic block diagram of a variation of the digital camerashown in FIG. 1.

FIG. 8 is a flowchart of a variation of the movement informationcalculation process in FIG. 3.

FIGS. 9A and 9B are diagrams useful in explaining movement informationused in the variation in FIG. 8.

DESCRIPTION OF THE EMBODIMENTS

The present invention will now be described in detail below withreference to the accompanying drawings showing embodiments thereof.

Although in the present embodiment, the invention is applied to adigital camera as an image processing apparatus, the invention is notlimitedly applied to the digital camera. That is, the invention can beapplied to any other image processing apparatus, insofar as it is animage processing apparatus that is capable of generating stereoscopicprint data for performing stereoscopic print processing.

FIG. 1 is a schematic block diagram of the digital camera, denoted byreference numeral 100, as the image processing apparatus according tothe embodiment of the invention.

Referring to FIG. 1, the digital camera 100 includes a photographingsection 101, an image processor 106, a memory controller 107, a systemcontroller 108, a memory 109, a nonvolatile memory 110, a displaysection 111, an operation section 112, an external interface 113, arecording medium interface 114, a ranging section 116, a distanceinformation-calculating section 117, a movement information-calculatingsection 118, a stereoscopic information-generating section 119, and aprint data-generating section 120. These components are interconnectedvia a bus 105. Further, a recording medium 115 and the recording mediuminterface 114 are connected to each other. The photographing section 101includes a photographic lens 102, an image pickup sensor 103, and an A/Dconverter 104.

The photographing section 101 acquires image information of an object.More specifically, in the photographing section 101, the photographiclens 102 takes in an optical image of the object, the image pickupsensor 103 converts the acquired image information to an analog imagesignal, and the A/D converter 104 converts the converted analog imagesignal to a digital image signal. The photographing section 101 sendsthe digital image signal converted by the A/D converter 104 to thememory 109 and the recording medium 115, and the memory 109 and therecording medium 115 store the received digital image signal as imagedata. In the present embodiment, when the digital camera 100 is set to aphotographing mode, preview processing is performed before the start ofphotographing to acquire a plurality of image information items from thephotographing section 101 at different times.

The image processor 106 acquires the image data from the memory 109, andperforms resize processing and color conversion processing on theacquired image data. Further, the image processor 106 performscalculation processing using the image information acquired by thephotographing section 101, and sends a result of calculation to thesystem controller 108. The memory controller 107 manages various datastored in the memory 109. The system controller 108 performs centralizedcontrol of the overall operation of the digital camera 100 by executingvarious programs stored in the nonvolatile memory 110. For example, thesystem controller 108 delivers control signals for performing exposurecontrol, ranging control, and optical control to the photographingsection 101, based on results of calculation received from the imageprocessor 106. The memory 109 stores the digital image signal sent fromthe photographing section 101 as image data. As the nonvolatile memory110, e.g., an electrically erasable programmable read-only memory(EEPROM) is used to store various programs and various settinginformation. The display section 111 displays, e.g., image informationacquired by preview processing. The operation section 112 sends varioussetting information set by a user's operation to the system controller108. The external interface 113 performs data communication with variousdevices connected thereto, e.g., via a USB connector (not shown). Therecording medium interface 114 performs data communication with therecording medium 115. The recording medium 115 is implemented, e.g., bya semiconductor memory or a magnetic disc. The memory 109 and therecording medium 115 store various data used in a movement informationcalculation process and a stereoscopic information calculation process,described hereinafter with reference to FIGS. 3 and 5, respectively.

The ranging section 116 performs ranging processing on the plurality ofimage information items, respectively, that have been acquired by thephotographing section 101 at different times, and sends ranginginformation acquired by the ranging processing to the distanceinformation-calculating section 117. In the ranging processing, forexample, an active method or a passive method is used. In the activemethod, for example, the ranging section 116 irradiates an object withlight, such as infrared light, and calculates the ranging informationbased on, e.g., a time period until the irradiated light is reflectedand returns to the ranging section 116. On the other hand, in thepassive method, the ranging section 116 calculates the ranginginformation based on the image information acquired by the photographingsection 101 without using infrared light or the like. The distanceinformation-calculating section 117 calculates the distance informationindicating a distance from the photographing section 101 to the objectbased on each image information item and the ranging informationassociated with the image information. The image information acquired bythe photographing section 101 is divided into a plurality of blocks eachhaving a predetermined size, and the distance information is calculatedwith respect to each divided block of the plurality of blocks. Themovement information-calculating section 118 calculates movementinformation of a moving body that is a moving object based ondifferences between distance information items that are associated withrespective image information items and calculated by the distanceinformation-calculating section 117. The movement information is formedby amounts of movement in three dimensions in X, Y, and Z directions,that are represented by a horizontal vector p, a vertical vector q, anda depth vector r, appearing in FIG. 2. The stereoscopicinformation-generating section 119 calculates stereoscopic informationindicating a thickness of an object image in a printing result based onthe distance information and the movement information. The printdata-generating section 120 generates stereoscopic print data based onthe image information and the stereoscopic information. For example, anSTL (Standard Triangle Language) or VRML (Virtual Reality ModelingLanguage) method is used for the stereoscopic print data.

Next, the movement information calculation process and the stereoscopicinformation calculation process in the present embodiment will bedescribed with reference to FIGS. 3 and 5, respectively.

In generation of the stereoscopic print data, although informationindicating protrusions and indentations of an object can be easilyacquired based on the distance information indicating the distance to anobject, it is difficult to acquire information indicating the track of amoving body directly from the distance information. Therefore,conventional image processing apparatuses cannot generate stereoscopicprint data that gives a printing result expressing a feeling of livelymovement of a moving body.

In contrast, in the present embodiment, the stereoscopic print data isgenerated based on the acquired image information, the distanceinformation indicating a distance to an object, and the movementinformation of a moving body.

FIG. 3 is a flowchart of the movement information calculation processperformed by the digital camera 100 shown in FIG. 1.

The information calculation process in FIG. 3 is performed by the systemcontroller 108 that executes various programs stored in the nonvolatilememory 110. Further, insofar as the image information and the movementinformation are stored in the memory 109 or the recording medium 115,the information calculation process in FIG. 3 can be performed any timeby acquiring the stored image information and movement information. Inthe present embodiment, however, the description will be given of a casein which the information calculation process is performed during a timeperiod between preview processing before photographing and thephotographing, performed by the digital camera 100, by way of example.

Referring to FIG. 3, first, the system controller 108 acquires aplurality of image information items at different times by the previewprocessing performed by the photographing section 101 (step S301). Then,the system controller 108 controls the distance information-calculatingsection 117 to calculate the distance information indicating a distancefrom the photographing section 101 to the object using each of the imageinformation items acquired in the step S301 (step S302). Morespecifically, for example, when the image information shown in FIG. 4Ais acquired, the distance information-calculating section 117 divides,as shown in FIG. 4B, the acquired image information into a plurality ofblocks by equally dividing the image information into eight blocks withrespect to an X-axis direction and six blocks with respect to a Y-axisdirection. Then, the distance information-calculating section 117acquires ranging information associated with each divided block from theranging section 116, and calculates the distance information indicatinga distance from the photographing section 101 to an object in each ofthe blocks based on the acquired ranging information. The distanceinformation is a distance with respect to the Z-axis direction in FIG.2. In the present embodiment, as shown in FIG. 4C, a distance Cindicates the largest distance from the photographing section 101, adistance B indicates the smallest distance from the photographingsection 101, and a distance A indicates a distance between the distanceC and the distance B. Note that symbols “A”, “B”, and “C” added to therespective blocks in FIG. 4B represent information items of the distanceA, B, and C of the blocks.

Then, the system controller 108 acquires the image information and thedistance information (the operation of a distance informationacquisition unit) and calculates the movement information of a movingbody based on the acquired image information and distance information(step S303) (the operation of a calculation unit). More specifically,first, the movement information-calculating section 118 identifies anobject area formed by blocks each including at least part of the objectbased on each of the image information items acquired in the step S301.In identifying the object area, for example, the movementinformation-calculating section 118 judges each block of which thedistance information calculated in the step S302 is within apredetermined range as a block forming the same object area. Further,the movement information-calculating section 118 identifies an areaformed by the blocks of which the distance information is within thepredetermined range as an object area. In the present embodiment, ahatched area in FIG. 4D corresponding to the image information acquiredin the step S301 that is formed by the blocks of which the distanceinformation calculated in the step S302 is within the predeterminedrange, i.e. an area formed by the blocks of which the distanceinformation is “A” or “B”, is identified as an object area 401.

Next, the movement information-calculating section 118 identifiescenter-of-gravity position information g (gx, gy, gz) of the identifiedobject area 401 (the operation of an identification unit). In theidentification of the center-of-gravity position information g, there isidentified center-of-gravity position information included in each ofimage information items acquired at different times, for example, thecenter-of-gravity position information g1 (g1 x, g1 y, g1 z) of theidentified object area 401 included in the image information of ann−1-th image (FIG. 4E) and the center-of-gravity position information g2(g2 x, g2 y, g2 z) of the identified object area 401 included in theimage information of an n-th image (FIG. 4F). The movement informationis calculated based on a difference in the identified center-of-gravityposition information g between the temporally sequential imageinformation items, for example, a difference between the identifiedcenter-of-gravity position information g1 (g1 x, g1 y, g1 z) andcenter-of-gravity position information g2 (g2 x, g2 y, g2 z), shown inFIG. 4G. The horizontal vector p of the movement information is adifference between g2 x and g1 x, the vertical vector q is a differencebetween g2 y and g1 y, and the depth vector r is a difference between g2z and g1 z. In the present embodiment, if there is a difference betweenthe center-of-gravity position information g1 and the center-of-gravityposition information g2, it is determined that the object area 401 is anarea indicative of a moving body (hereinafter referred to as the “movingbody area”), whereas, if there is no difference between thecenter-of-gravity position information g1 and the center-of-gravityposition information g2, it is determined that the object area 401 isnot an area indicative of a moving body. Note that, in the presentembodiment, the movement information may be calculated by calculating anarithmetic mean of the center-of-gravity position information g of theobject area included in the image information of three or moretemporally sequential images.

Then, the system controller 108 determines whether or not a managementdata generation instruction is received, for generating management dataincluding information indicating association of the image information ofthe n-th image and the distance information and movement informationassociated with the n-th image information (step S304).

If it is determined in the step S304 that the management data generationinstruction has been received, the system controller 108 generates themanagement data (step S305). Then, the system controller 108 stores theimage information, distance information, movement information, andmanagement data, described above, in the memory 109 or the recordingmedium 115 (step S306), followed by terminating the present process.

If it is determined in the step S304 that the management data generationinstruction has not been received, the system controller 108 returns tothe step S301.

According to the information calculation process described above withreference to FIG. 3, the movement information is calculated based on adifference between the center-of-gravity position information g1 (g1 x,g1 y, g1 z) and the center-of-gravity position information g2 (g2 x, g2y, g2 z) of the object area 401 included in the n−1-th image and then-th image, respectively. This makes it possible to reduce theinformation used for the movement information calculation process to anecessary minimum, and thereby reduce the load of the movementinformation calculation process.

Although in the above-described information calculation process in FIG.3, the n-th image information corresponds to the last image informationacquired in the preview processing, the image information from which themovement information is calculated by the information calculationprocess in FIG. 3 is not limited to the n-th image information. Forexample, the image information used for calculation of the movementinformation may be image information acquired at any given time in thepreview processing.

FIG. 5 is a flowchart of the stereoscopic information calculationprocess performed by the digital camera 100 shown in FIG. 1.

The stereoscopic information calculation process in FIG. 5 is performedby the system controller 108 that executes various programs stored inthe nonvolatile memory 110. Note that the stereoscopic informationcalculation process in FIG. 5 is performed during a time period afterthe information calculation process in FIG. 3 is terminated and beforethe stereoscopic print processing is started.

Referring to FIG. 5, first, the system controller 108 acquires the imageinformation, the distance information, the movement information, and themanagement data, stored in the step S306 in FIG. 3 (step S501). Then,the system controller 108 causes the stereoscopic information-generatingsection 119 to calculate stereoscopic information B into which a factorof movement of a moving body is incorporated, based on the distanceinformation, the movement information, and the management data, acquiredin the step S501 (step S502). More specifically, first, the systemcontroller 108 calculates stereoscopic information A that is informationindicating protrusions and indentations of an object, based on theacquired distance information. The stereoscopic information A is formedby items of position information in three dimensions associated with theX-axis, the Y-axis, and the Z-axis, respectively (i.e. an X component, aY component, and a Z component of a position vector). Further, thesystem controller 108 adds information providing a feeling of livelymovement of a moving body to the stereoscopic information A, using thehorizontal vector p, the vertical vector q, and the depth vector r ofthe acquired movement information. More specifically, the systemcontroller 108 determines the Z component of the stereoscopicinformation A such that the Z component is increased as the scalaramounts of the horizontal vector p and the vertical vector q of themovement information are larger (the position information associatedwith the Z axis is offset) (the operation of a determination unit), tothereby generate the stereoscopic information B. That is, in the presentembodiment, the two-dimensional movement of the moving body (movement ofthe moving body associated with the X-axis and movement associated withthe Y-axis) is converted to a thickness of the object image. Forexample, in FIG. 6, the distance information of each block of the objectarea 401 that is a moving body area is set to “h1” to which thethickness is given according to the two-dimensional movement of themoving body, and the distance information of each block of other thanthe object area 401 is set to “h0” to which the thickness is not given.Although in the present embodiment, the Z component of the stereoscopicinformation A is increased only according to the scalar amounts of thehorizontal vector p and the vertical vector q of the movementinformation, the Z component of the stereoscopic information A may beincreased not only according to the scalar amounts of the horizontalvector p and the vertical vector q of the movement information, but alsoaccording to a scalar amount of the depth vector r.

Then, the system controller 108 controls the print data-generatingsection 120 to generate the stereoscopic print data based on the imageinformation acquired in the step S501 and the stereoscopic information Bcalculated in the step S502 (step S503) (the operation of a generationunit). Then, the system controller 108 stores the generated stereoscopicprint data in the memory 109 or the recording medium 115 (step S504),followed by terminating the present process.

According to the above-described information calculation process in FIG.3 and the stereoscopic information calculation process in FIG. 5, thestereoscopic print data is generated based on the image information, thedistance information, and the movement information. This makes itpossible to process the stereoscopic information A that is generatedbased on the distance information indicating a distance to the object,based on the movement information indicating movement of a moving body,and hence, it is possible to generate the stereoscopic print data towhich the factor of the movement of the moving body is incorporated,based on the processed stereoscopic information B. That is, it ispossible to generate the stereoscopic print data that gives a printingresult expressing a feeling of lively movement of an object.

Further, according to the information calculation process and thestereoscopic information calculation process described above withreference to FIGS. 3 and 5, respectively, as the scalar amount of eachvector of the movement information is larger, the Z component of thestereoscopic information A is increased, whereby the thickness of theobject image is increased. As a result, as the speed of movement of amoving body is higher, the object image in the printing result appearscloser to a viewer, and hence, it is possible to generate thestereoscopic print data that enables a viewer to easily visuallyrecognize the speed of the movement of the moving body.

In the above-described stereoscopic information calculation process inFIG. 5, an increased amount of the Z component in the stereoscopicinformation A, determined based on at least one of the horizontal vectorp, the vertical vector q, and the depth vector r of the movementinformation that is desired by the user to emphasize the movement of theobject, may be defined to be larger than an increased amount in thestereoscopic information A that is determined based on the othervectors. As a result, for example, by making an increased amount of theZ component in the stereoscopic information A, determined based on thehorizontal vector p of the vectors of the stereoscopic information,larger than an increased amount in the stereoscopic information A,determined based on the other vectors, it is possible to generate thestereoscopic print data that emphasizes movement of the moving body inthe X direction.

The embodiment of the invention has been described heretofore, but theinvention is not limited to the above-described embodiment.

For example, although in the above-described embodiment, the movementinformation of an object is calculated based on a difference incenter-of-gravity position information g of the object area between aplurality of image information items, the movement information of anobject may be calculated based on a difference in distance informationof part of the object in each divided block between the imageinformation items.

FIG. 7 is a schematic block diagram of a variation of the digital camera100 shown in FIG. 1.

This variation basically has the same configuration and performs thesame operation as the above-described embodiment. Therefore, redundantdescription of the same configuration and same operation is omitted, andthe following detailed description will be given only of the differentconfiguration and the different operation from the above-describedembodiment.

Referring to FIG. 7, a digital camera 700 includes not only thecomponents of the digital camera 100, but also an in-plane movementamount-calculating section 701. The in-plane movement amount-calculatingsection 701 is also connected to the other components via the bus 105.

The in-plane movement amount-calculating section 701 calculates anin-plane movement vector based on a plurality of image information itemsacquired by the photographing section 101 at different times. Morespecifically, the in-plane movement amount-calculating section 701divides each acquired image information into a plurality of blocks, andcalculates an in-plane movement vector of predetermined part of anobject in each divided block.

FIG. 8 is a flowchart of a variation of the movement informationcalculation process in FIG. 3.

The movement information calculation process in FIG. 8 is performed bythe system controller 108 that executes various programs stored in thenonvolatile memory 110. The movement information calculation process inFIG. 8 is also performed during a time period between preview processingbefore photographing and the photographing, performed by the digitalcamera 700, by way of example.

Referring to FIG. 8, first, the same processing as in the steps S301 andS302 in FIG. 3 is performed. Then, the system controller 108 controlsthe in-plane movement amount-calculating section 701 to calculate thein-surface movement vector in each block in the image information (stepS801). More specifically, the in-plane movement amount-calculatingsection 701 detects position information F1 (F1 x, F1 y) of part E of anobject in each block of the n−1-th image information, for example, in ablock D appearing in FIG. 9A. The position information F1 (F1 x, F1 y)is formed by position information of two dimensions represented by theX-axis and the Y-axis in FIGS. 9A and 9B. Then, the in-plane movementamount-calculating section 701 detects position information F2 (F2 x, F2y) of the part E of the object in the block D of the n-th imageinformation, and calculates an in-plane movement vector of the block Dbased on a difference between the position information F1 and theposition information F2. The in-plane movement amount-calculatingsection 701 performs the same processing with respect to the otherblocks, whereby the in-plane movement vectors of the respective blocksare calculated.

Then, the system controller 108 calculates the movement information ofthe object based on the in-plane movement vectors calculated in the stepS801 and the distance information acquired in the step S302 (step S802).In the step S802, the X and Y components of the in-plane movement vectorcalculated in the step S801 and the X and Y components of the positioninformation in the X-axis and Y-axis directions in the distanceinformation are synthesized to thereby calculate the X and Y components(the horizontal vector p and the vertical vector q) of the movementinformation. Further, the position information in the Z-axis directionin the distance information becomes a Z component (depth vector r) ofthe movement information. Then, the system controller 108 performs thesame processing as the steps S304 to S306 in FIG. 3, followed byterminating the present process. Note that the in-plane movement vectoris not calculated in a block where the part E of the object is notincluded, and as a result, the movement information is not calculatedeither.

Although, in the above-described movement information calculationprocess in FIG. 8, the n-th image information corresponds to the lastimage information acquired in the preview processing, the imageinformation from which the movement information is calculated using themovement information calculation process in FIG. 8 is not limited to then-th image information. For example, the image information used forcalculation of the movement information may be image informationacquired at any given time in the preview processing.

According to the above-described movement information calculationprocess in FIG. 8, the n−1-th image information and the n-th imageinformation, acquired in the preview processing, are divided into aplurality of blocks, respectively. Further, in the n−1-th imageinformation and the n-th image information, the in-plane movementvectors of the divided blocks included in the n-th image information arecalculated based on a difference between the position information F1 andthe position information F2 of the part E of the object in each dividedblock. As a consequence, the in-plane movement vectors are calculated inthe respective blocks, and hence, by partially processing thestereoscopic print data according to each in-plane movement vector, itis possible to cause the movement of part of the object associated witheach block to be individually reflected on a printing result, making itpossible to provide a clear expression of the movement of each part ofthe object. As a result, it is possible to generate stereoscopic printdata that gives a printing result expressing a feeling of livelymovement of an object more.

More specifically, in each block including the part E of the object, thestereoscopic information of the part E of the object is generated basedon the movement information of the corresponding block. That is, as thescalar amounts of the horizontal vector p and the vertical vector q inthe movement information of each block are larger, the Z component ofthe stereoscopic information of the part E of the object is increased,and the two-dimensional movement of the part E of the object isconverted to a partial thickness of the object image on a block-by-blockbasis. For example, in FIG. 9B, the distance information of blocksincluding the part E of the object is set to “h1” to which the thicknessis added according to the two-dimensional movement of the part E of theobject in the corresponding blocks. Further, the distance information ofthe other blocks including the part E of the object is set to “h2” towhich the thickness is added according to the two-dimensional movementof the part E of the object in the corresponding other blocks.

Although in the above-described movement information calculation processin FIG. 8, when calculating the movement information of the object, theZ component of the movement information is calculated using the distanceinformation, the movement information of the object may be calculatedusing only the in-plane movement vector. That is, it is not necessarilyrequired to calculate the Z component of the movement information. Thismakes it possible to calculate the movement information of an object,even when the distance information of the object cannot be acquired orthe accuracy of calculation of the distance information is low, due tothe photographing condition, whereby it is possible to generatestereoscopic print data that gives a printing result expressing afeeling of lively movement of an object.

Other Embodiments

Embodiments of the present invention can also be realized by a computerof a system or an apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (that may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiments and/or thatincludes one or more circuits (e.g., an application specific integratedcircuit (ASIC)) for performing the functions of one or more of theabove-described embodiments, and by a method performed by the computerof the system or the apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiments and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiments. Thecomputer may comprise one or more processors (e.g., a central processingunit (CPU), or a micro processing unit (MPU)) and may include a networkof separate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), a digital versatile disc (DVD), or a Blu-ray Disc(BD)™), a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

What is claimed is:
 1. An image processing apparatus comprising: (A) aphotographing section that acquires a first image of an object and asecond image of the object, the first image and the second image beingacquired at different times, and each including an object image of theobject and a background portion; (B) at least one memory that storesinstructions; and (C) at least one processor coupled to the at least onememory, and configured to execute the instructions to cause the imageprocessing apparatus to function as: (a) a distance informationacquisition unit configured to acquire distance information associatedwith the object image included in each of the first image and the secondimage, the distance information corresponding to a distance in a depthdirection of the object, and indicating a distance from thephotographing section to the object; (b) a movement vector calculationunit configured to calculate a movement vector of the object image in ahorizontal direction, a vertical direction, and the depth direction,based on position information of the object image included in each ofthe first image and the second image; (c) a thickness determination unitconfigured to determine a thickness of the object image, in the depthdirection, included in each of the first image and the second imageaccording to at least one of a horizontal component and a verticalcomponent of the movement vector; (d) a generation unit configured togenerate stereoscopic print data based on (i) the second image, (ii) thedistance information associated with the object image included in thesecond image and acquired by the distance information acquisition unit,(iii) the movement vector calculated by the movement vector calculationunit, and (iv) the determined thickness of the object image, byincreasing a depth component of the position information of the objectimage included in the second image based on the at least one of thehorizontal component and the vertical component of the movement vector,wherein the generation unit increases the thickness of the object imagebased on an increase of a scalar amount of the at least one of thehorizontal component and the vertical component of the movement vector,without increasing a thickness of the background portion of the secondimage; and (e) a storing unit that stores the generated stereoscopicprint data in one of a second memory and a recording medium.
 2. Theimage processing apparatus according to claim 1, wherein the imageprocessing apparatus further functions as (f) an identification unitconfigured to identify center-of-gravity position information of theobject image included in each of the first image and the second image,and wherein the movement vector calculation unit calculates the movementvector based on a difference in the center-of-gravity positioninformation of the object image included in the first image and thecenter-of-gravity position information of the object image included inthe second image.
 3. The image processing apparatus according to claim1, wherein the movement vector calculation unit divides each of thefirst image and the second image into a plurality of blocks, andcalculates the movement vector of the object image included in thesecond image, based on a difference in position of part of the objectimage corresponding to each of the blocks between the first image andthe second image.
 4. The image processing apparatus according to claim1, wherein the movement vector includes the horizontal component, thevertical component, and a depth component, and wherein the thicknessdetermination unit determines the thickness of the object image based onboth of the horizontal component and the vertical component of themovement vector.
 5. A method of controlling an image processingapparatus, the method comprising: acquiring a first image of an objectand a second image of the object, the first image and the second imagebeing acquired at different times, and each including an object image ofthe object and a background portion; acquiring distance informationassociated with the object image included in each of the first image andthe second image, the distance information corresponding to a distancein a depth direction of the object, and indicating a distance from thephotographing section to the object; calculating a movement vector ofthe object image, at least in a horizontal direction, a verticaldirection, and the depth direction, based on position information of theobject image included in each of the first image and the second image;determining a thickness of the object image, in the depth direction,included in each of the first image and the second image according to atleast one of a horizontal component and a vertical component of themovement vector; generating stereoscopic print data based on (i) thesecond image, (ii) the acquired distance information associated with theobject image included in the second image, (iii) the calculated movementvector, and (iv) the determined thickness of the object image, byincreasing a depth component of the position information of the objectimage included in the second image based on the at least one of thehorizontal component and the vertical component of the movement vector,wherein, during the generation of the stereoscopic print data, thethickness of the object image is increased based on an increase of ascalar amount of the at least one of the horizontal component and thevertical component of the movement vector, without increasing athickness of the background portion of the second image; and storing, inone of a memory and a recording medium, the generated stereoscopic printdata.
 6. The method according to claim 5, further comprising identifyingcenter-of-gravity position information of the object image included ineach of the first image and the second image, wherein the movementvector is calculated based on a difference in the center-of-gravityposition information of the object image included in the first image andthe center-of-gravity position information of the object image includedin the second image.
 7. The method according to claim 5, wherein, duringthe calculating step, each of the first image and the second image isdivided into a plurality of blocks, and the movement vector of theobject image included in the second image is calculated based on adifference in position of part of the object image corresponding to eachof the blocks between the first image and the second image.
 8. Themethod according to claim 5, wherein the movement vector includes thehorizontal component, the vertical component, and a depth component, andthe thickness of the object image is determined based on both of thehorizontal component and the vertical component of the movement vector.9. A non-transitory computer-readable storage medium storing acomputer-executable program for executing a method of controlling animage processing apparatus, wherein the method comprises: acquiring afirst image of an object and a second image of the object, the firstimage and the second image being acquired at different times, and eachincluding an object image of the object and a background portion;acquiring distance information associated with the object image includedin each of the first image and the second image, the distanceinformation corresponding to a distance in a depth direction of theobject, and indicating a distance from the photographing section to theobject; calculating a movement vector of the object image, at least in ahorizontal direction, a vertical direction, and the depth direction,based on position information of the object image included in each ofthe first image and the second image; determining a thickness of theobject image, in the depth direction, included in each of the firstimage and the second image according to at least one of a horizontalcomponent and a vertical component of the movement vector; generatingstereoscopic print data based on (i) the second image, (ii) the acquireddistance information associated with the object image included in thesecond image, (iii) the calculated movement vector, and (iv) thedetermined thickness of the object image, by increasing a depthcomponent of the position information of the object image included inthe second image based on the at least one of the horizontal componentand the vertical component of the movement vector, wherein, during thegeneration of the stereoscopic print data, the thickness of the objectimage is increased based on an increase of a scalar amount of the atleast one of the horizontal component and the vertical component of themovement vector, without increasing a thickness of the backgroundportion of the second image; and storing, in one of a memory and arecording medium, the generated stereoscopic print data.
 10. Thenon-transitory computer-readable storage medium according to claim 9,wherein the method further comprises identifying center-of-gravityposition information of the object image included in each of the firstimage and the second image, and wherein the movement vector iscalculated based on a difference in the center-of-gravity positioninformation of the object image included in the first image and thecenter-of-gravity position information of the object image included inthe second image.
 11. The non-transitory computer-readable storagemedium according to claim 9, wherein, during the calculating step, eachof the first image and the second image is divided into a plurality ofblocks, and the movement vector of the object image included in thesecond image is calculated based on a difference in position of part ofthe object image corresponding to each of the blocks between the firstimage and the second image.
 12. The non-transitory computer-readablestorage medium according to claim 9, wherein the movement vectorincludes the horizontal component, the vertical component, and a depthcomponent, and the thickness of the object image is determined based onboth of the horizontal component and the vertical component of themovement vector.